Quantitative vs. Qualitative: Envisioning a 'Partnership' in Educational Research

To the Editor:

The exchange between Tony Wagner ("Beyond Testing: The 7 Disciplines for Strengthening Instruction," Commentary, Nov. 12, 2003) and Patrick Groff "Qualitative Data and The Eye of the Beholder," Letters, Dec. 3, 2003) underscores the long- standing friction between advocates of quantitative and qualitative research. Most advocates seem to regard the two approaches as mutually exclusive. Yet as someone who firmly believes in the use of data to improve education, I think the right kind of qualitative research could make quantitative studies far more valuable.

With a bit of regression analysis, I can identify those schools that outperform their peers with essentially similar students. By repeating that analysis for additional tests and additional schools, I can identify those schools that consistently outperform their peers over time and over multiple subjects. By looking at how their students scored when they moved on to the next level of schooling, I can even check that the schools' scores weren't artificially inflated.

What I cannot do easily is identify the reasons for those schools' performance. Schools are notoriously prone to mislabel their educational practices, in part for political reasons and in part because they are often genuinely unaware of how to accurately describe their practices. There is even less reliable information on what the teachers do or how the schools are led.

Starting about 40 years ago, a series of studies seemed to show that educational outcomes were largely driven by student socioeconomic status. The researchers also determined that the schools had little effect. None of the school factors the researchers looked at—what one study labeled the "accreditation" factors because those were the things accrediting agencies counted—seemed to make a significant difference in student outcomes.

In retrospect, however, it appears that the researchers were hobbled by the limitations of their data. It is becoming increasingly clear that schools can have a major impact on student achievement, but these impacts have far more to do with teaching practices, the program used, and school leadership than with the accreditation factors. Unfortunately for the researchers, they were stuck with the accreditation factors because the more important factors were not measured.

Qualitative research that accurately categorized teaching, educational programs, and leadership at schools could be invaluable in finding the relationships between these and student achievement. For example, if it were desired to compare the effect of two different math programs, qualitative research could be used to assure that the classes in the two samples actually followed the program they purported to follow.

Perhaps I am being overly optimistic in envisioning such a partnership between quantitative and qualitative researchers. The qualitative researchers I know are very ideological, viewing themselves as advocates of particular practices, rather than dispassionate analysts. Still, the value could be great.

Bruce Thompson
Professor of Management
Milwaukee School of Engineering
Milwaukee, Wis.